Image fusion has been found useful in many applications. A single recorded image from an image sensor may contain insufficient details of a scene due to the incompatibility between the image sensor's capture range and the characteristics of the scene. For example, because a natural scene can have a high dynamic range (HDR) that exceeds the dynamic range of an image sensor, a single recorded image is likely to exhibit under- or over-exposure in some regions, which leads to detail loss in those regions. Image fusion can solve such problems by combining local details from a plurality of images recorded by an image sensor under different settings of an imaging device, such as under different exposure settings, or from a plurality of images recorded by different image sensors, each of which captures some but not all characteristics of the scene.
One type of image fusion method known in the art is based on multi-scale decomposition (MSD). Two types of commonly used MSD schemes include pyramid transform, such as Laplacian pyramid transform, and wavelet transform, such as discrete wavelet transform. Images are decomposed into multi-scale representations (MSRs), each of which contain an approximation scale generated by low-pass filtering and one or more detail scales generated by high-pass or band-pass filtering. The fused image is reconstructed by inverse MSD from a combined MSR.
Another type of image fusion method known in the art computes local features at the original image scale, and then, by solving an optimization problem, generates the fused image or the fusion weights, which are to be used as weighting factors when the images are linearly combined. Another type of image fusion methods divides images into blocks and generates a fused image by optimizing one or more criteria within each block.
Another type of method that can achieve a similar effect, as image fusion methods do when fusing images taken under different exposure settings, is the two-phase procedure of HDR reconstruction and tone mapping. An HDR image is reconstructed from the input images, and then the dynamic range of this HDR image is compressed in the tone mapping phase. However, the above types of methods may impose high spatial computational cost and/or high temporal computational cost, or introduce artifacts into a fused image due to non-linear transformations of pixel values or due to operations performed only in small local regions.
Accordingly, what is needed is a method and system that effectively and efficiently combines useful information from images, especially in the case of fusing images taken under different exposure settings.